Hierarchical Knowledge-Gradient for Sequential Sampling

We consider the problem of selecting the best of a finite but very large set of alternatives. Each alternative may be characterized by a multi-dimensional vector and has independent normal rewards. This problem arises in various settings such as (i) ranking and selection, (ii) simulation optimization where the unknown mean of each alternative is estimated with stochastic simulation output, and (iii) approximate dynamic programming where we need to estimate values based on Monte-Carlo simulation. We use a Bayesian probability model for the unknown reward of each alternative and follow a fully sequential sampling policy called the knowledge-gradient policy. This policy myopically optimizes the expected increment in the value of sampling information in each time period. Because the number of alternatives is large, we propose a hierarchical aggregation technique that uses the common features shared by alternatives to learn about many alternatives from even a single measurement, thus greatly reducing the measurement effort required. We demonstrate how this hierarchical knowledge-gradient policy can be applied to efficiently maximize a continuous function and prove that this policy finds a globally optimal alternative in the limit

Dynamic multi-period freight consolidation

Logistic Service Providers (LSPs) o ering hinterland transportation face the trade-o between e ciently using the capacity of long-haul vehicles and minimizing the rst and last-mile costs. To achieve the optimal trade-o , freights have to be consolidated considering the variation in the arrival of freight and their characteristics, the applicable transportation restrictions, and the interdependence of decisions over time. We propose the use of a Markov model and an Approximate Dynamic Programming (ADP) algorithm to consolidate the right freights in such transportation settings. Our model incorporates probabilistic knowledge of the arrival of freights and their characteristics, as well as generic de finitions of transportation restrictions and costs. Using small test instances, we show that our ADP solution provides accurate approximations to the optimal solution of the Markov model. Using a larger problem instance, we show that our modeling approach has signi cant bene ts when compared to common-practice heuristic approaches

The delivery dispatching problem with time windows for urban consolidation centers

This paper addresses the dispatch decision problem faced by an urban consolidation center. The center receives orders according to a stochastic arrival process, and dispatches them for the last-mile distribution in batches. The operator of the center aims to fi nd the cost-minimizing consolidation policy, depending on the orders at hand, pre-announced orders, and stochastic arrivals. We present this problem as a variant of the Delivery Dispatching Problem that includes dispatch windows, and model it as a Markov decision problem. For toy-sized instances, we solve this model to optimality. Through numerical experiments on these instances, we show that we approximate the optimal values with an error of less than 2%. Larger instances suff er from intractably large state-, outcome-, and action spaces. We propose an Approximate Dynamic Programming (ADP) algorithm that can handle such instances, using value function approximation to estimate the downstream costs. To cope with large action spaces - with sizes up to 2120 in our experiments - we formulate an integer linear program to be used within our ADP algorithm. To evaluate the performance of our ADP policies, we test against various benchmark policies, including a lookahead policy based on scenario sampling. We test the performance of ADP on a variety of networks. When the dispatching problem provides su fficient fl+6exibility in dispatch times, ADP outperforms our myopic benchmark policies by more than 15%, and lookahead policies by over 10%

Interaction between intelligent agent strategies for real-time transportation planning

In this paper we study the real-time scheduling of time-sensitive full truckload pickup-and-delivery jobs. The problem involves the allocation of jobs to a fixed set of vehicles which might belong to different collaborating transportation agencies. A recently proposed solution methodology for this problem is the use of a multi-agent system where shipper agents offer jobs through sequential auctions and vehicle agents bid on these jobs. In this paper we consider such a system where both the vehicle agents and the shipper agents are using profit maximizing look-ahead strategies. Our main contribution is that we study the interrelation of these strategies and their impact on the system-wide logistical costs. From our simulation results, we conclude that the system-wide logistical costs (i) are always reduced by using the look-ahead strategies instead of a myopic strategy (10–20%) and (ii) the joint effect of two look-ahead strategies is larger than the effect of an individual strategy. To provide an indication of the savings that might be realized under centralized decision making, we benchmark our results against an integer programming approach